The Lifecycle Has Already Flipped

Walk into almost any engineering organization today, and you’ll find developers who describe a feature in prose and watch an agent implement it. Not a snippet, not an autocomplete: a route, a migration, a handler, tests, and a commit. The interesting part isn’t that the code gets written. It’s that the human is no longer in the authoring path. They’re at the edges: they prompt at the start, and they review (maybe) at the end.

Here’s the uncomfortable part most teams haven’t faced yet: if an agent built the app, a human can’t realistically pentest it at the rate it changes. A full application pentest is a multi-day, expensive, human-driven exercise. It is necessary, but when the app is regenerated or extended dozens of times a day, a human pentest of the whole thing is obsolete before the report is formatted. The only actor that can keep pace with an agent that writes code is another agent that attacks it.

Machine-Speed Code, Calendar-Speed Security

The core problem is a clock mismatch. Code is now authored continuously, by agents, all day. The audit that’s supposed to catch their mistakes runs, in most organizations, four times a year. Everything between those audits is uncovered surface area, and that gap is exactly where breaches live.

Table 1. Comparative numbers of AI agent introduced changes vs. human security testing.

Look at those numbers together. Hundreds of changes a day against one security snapshot a quarter. Thousands of commits sit between two audits. Any one of which can introduce an injection, a missing authorization check, or a leaked internal error that ends up costing millions. The remediation clock (weeks, even for known exploited vulnerabilities) tells you the industry is already losing the race on a human cadence. Adding agent-generated volume on top doesn’t strain that model. It ends it.

The takeaway: security has to run at the speed and time the code is written, per change, not per quarter. And “per change” at the agent volume means no human in the critical path. That’s the design decision the rest of this article is really about.

Why Is the Future Pentester in the CI to Test the Diff, not the Entire App

When people first hear “autonomous pentesting in CI,” they imagine pointing an AI at the whole running application on every commit and letting it rip. That instinct is wrong, and understanding why it’s wrong is the whole game.

Pentesting an entire application on every change is slow, expensive, and noisy. A full attack surface takes a long time to crawl and fuzz; the cost per run balloons; and worst of all, the signal-to-noise ratio collapses. You get a flood of findings against code nobody just touched, most of which are either already known, already accepted, or false positives. The one finding that actually matters, the one in the code that just changed, drowns in the pile. Do that on every commit, and engineers will mute the gate within a week. A security control that is muted is worse than no control because it creates false confidence.

The alternative is to make the Pentester behave like a good code reviewer: it looks at the diff. An agent that reads exactly what changed can reason about the specific new attack surface that the change introduced, attack only that, verify only that, and hand back a small set of high-confidence findings scoped to code that is, by definition, fresh in the author’s mind. This is what makes autonomous pentesting fast enough to gate on and accurate enough to trust.

To be clear, the in-CI Pentester doesn’t replace the whole-application pentest; it protects it. When a prompt builds a whole feature, attacking each change as it is built catches the obvious issues at the source and keeps the periodic full pentest from drowning in a backlog it can no longer triage. The agent handles per-change volume; the human-driven pentest still covers the depth and cross-application flaws that a single diff can’t see.

Whole app, every commit – minutes-to-hours per run, cost scaling with total surface area, endless re-reporting of known issues, and a real finding lost in the noise until the gate gets muted.

Diff-scoped, autonomous – seconds-to-minutes, cost scaling with the size of the change, findings mapped to code the author just wrote, and high signal because verify-and-drop kills false positives. White-box reads the diff; black-box proves it live.

This is the inversion in one line: turn the pentest from a calendar event into a pipeline stage. Not scheduled quarterly by humans with weeks of lead time, but triggered per change. Not testing a frozen snapshot the app has already moved past, but testing exactly the diff that was just written. Not producing a PDF nobody re-runs, but emitting machine-readable fix instructions. And critically, it scales, because the attacker is also an agent. If an agent can write the code, an agent can attack it at the same volume.

I built a working system to prove this isn’t a slide. It’s called AutoPentest Loop. Here’s how it’s put together.

Eight Agents, One Graph, One State Object

The system takes one prompt in and produces a secured release out. In between, eight single-purpose agents run in a closed loop: develop the feature, build it, pentest it, fix what’s found, review the fix, rebuild, validate the fix holds, and report. Bounded retry loops wrap the fragile parts (Fig. 1).

Fig. 1: One prompt in, a secured release out. The orchestrator knows the workflow but nothing about the work.

The guiding principle is the separation of concerns taken to an extreme. The orchestrator knows the workflow but knows nothing about the work. It coordinates; it doesn’t compute. Each of the eight agents does exactly one thing and reports back. Agents never call each other; they read from a shared state object, do their one job, and write back. That isolation is the entire reason failures are debuggable: when something goes wrong, exactly one agent owns it.

The eight agents

• Developer: implements the requested feature (Claude Code, headless).
• Builder: builds, runs, and health-checks the app (Python + Docker).
• Build-Fixer: a recovery agent that repairs broken builds (Claude Code, headless). On the happy path, it never runs.
• Pentester: the hybrid white + black-box attacker (Claude API + tool-use). More on this below.
• Fixer: applies the Pentester’s fix prompts, one finding at a time (Claude Code, headless).
• Reviewer: judges whether a fix is correct, read-only, no file access (Claude API).
• Validator: re-runs the exploit and scans for regressions (Pentester components, scoped).
• Reporter: aggregates everything into Markdown / HTML / PDF (Claude API + templates).

The Orchestrator Is a State Machine, not a Script

The whole process runs on LangGraph deliberately, because a script would have been the wrong tool. There are three pieces:

• A state object: one Pydantic model, RunState, that is the single source of truth for a run.
• A graph of nodes where each node is a function that calls exactly one agent, and the edges between them are routing decisions.
• An execution engine that walks the graph and persists state to SQLite after every node, kills it mid-run, and resumes from the last checkpoint.

Listing 1

graph.py

# Edges are where routing — and safety — decisions get made.
g.add_conditional_edges("build", after_build, {
    "pentest": pentest,      # healthy build → attack the diff
    "build_fix": build_fix,  # broken build → recovery agent
    "report": report,        # give up cleanly → real failure report
})

The graph is the workflow. Every conditional edge is a place where a routing or safety decision happens. As we’ll see, it’s also where subtle bugs hide.

RunState: the Object You’ll Reference a Hundred Times

Nothing lives in an agent’s “head”. Everything that happens during a run is serialized into RunState: the prompt, the current phase, per-phase outputs, the findings, the fix attempts, and the retry counters that cause the loops to terminate.

Listing 2

state.py
class RunState(BaseModel):
    run_id, prompt, started_at
    current_phase: Phase

    # per-phase outputs
    developer_output, build_result
    findings: list[Finding]
    fix_attempts: list[FixAttempt]

    # retry counters — these are what enforce STOP
    build_attempts: int = 0
    fix_attempts_per_finding: dict
    review_reworks_per_finding: dict
    unfixable_findings: list[str]

    # bounds (config)
    max_build_attempts = 3

Four properties fall out of this design:

• It’s the single source of truth, no private state anywhere.
• It’s checkpointed every node, crash-safe, resumable, and fully auditable.
• The counters live here; they, plus unfixable_findings, are what make every loop terminate.
• Itbecomes the report; the Reporter renders this object verbatim: timeline, findings, fixes, verdicts.

Inside the Agentic Pentester

The Pentester is where the “test the diff” philosophy becomes concrete. It’s not one call to a model; it’s five internal stages. Moreover, it’s hybrid by design: it reads the diff (white-box) to know where to look, then actually exploits the running app (black-box) to prove the vulnerability is real (Fig. 2).

Fig. 2: The Pentester has five stages, not one. Only stage 4, verify-and-drop, separates a real gate from a noise generator.

• PLAN reads the diff and emits attack hypotheses as JSON, each mapped to an OWASP category and a CWE.
• RECON uses ZAP to spider and passively scan, then filters the URL set down to just the endpoints that the change touched.
• ATTACK is a tool-use loop where the model drives real tools to exploit each hypothesis.
• VERIFY re-runs every apparent exploit and drops anything that doesn’t reproduce. This is the false-positive killer, and it’s the difference between a gate engineer’s trust and one they mute.
• FIX-PROMPT writes a concrete, file-specific patch instruction for each finding that survived verification.

The Attack Loop: an Agent With Real Tools and Hard Caps

Inside ATTACK, the model isn’t hallucinating vulnerabilities; it’s orchestrating industry-standard tooling through tool use, and it’s on a leash.

Listing 3

pentester/attack.py
for iteration in range(MAX_ITERATIONS):        # cap #1: 30 iterations
    if elapsed > TIME_BUDGET:               # cap #2: 10-minute wall clock
        signal_time_up()
    resp = client.messages.create(
        tools=[nuclei, sqlmap, http_fuzz, record],
    )
    if resp.stop_reason == "end_turn":
        break

Four tools, one objective. nuclei_scan for known CVEs and misconfigurations. sqlmap_scan for SQL injection only (it’s slow, so it’s scoped tightly). http_fuzz for XSS, command injection, path traversal, IDOR, and SSRF. And record_finding, which is how the agent registers a confirmed, reproducible vulnerability. Two hard caps, 30 iterations, and a ten-minute budget guarantee the loop can’t run forever, even if the model wants to keep poking.

What it Finds on a Real Diff

Take the example feature prompt from Figure 1, a GET /admin-stats route with a filter parameter interpolated straight into raw SQL. That prompt is a trap on purpose. Here’s what a real run produced: each finding reproduced, with a fix prompt attached.

Table 2. Summary of the results.

The economics are the headline: roughly 60-90 seconds per pentest cycle and $1.40-$2.00 in API spend, with false positives dropped before they ever reach a human. That is what “fast and accurate enough to gate on” costs in practice, and it’s only that cheap because the agent tested the diff, not the app.

Fix, Review, Prove it – With Two Different Judges

Finding a vulnerability is half the job; proving it’s gone is the other half. Three agents share that work, and the split between them is deliberate. The Fixer follows the fix prompt verbatim, applies only the fix (no refactors), and commits each patch. The Reviewer (Opus, read-only, no file access) asks three questions: does it fix the root cause, does it introduce new issues, and does the feature still work? Its verdict is approved, needs_rework, or rejected. Then the Validator re-runs the original exploit against the running app: still works means still_vulnerable; gone triggers a focused regression scan, then marks it fixed.

Autonomy Needs Bounds: Every Loop Must Stop

Here is the single most important engineering lesson from building this. A self-healing loop with no bounds is a self-funding bug. An agent that can retry a fix is wonderful right up until the fix never converges, at which point it retries forever, burning money and time while reporting that it’s “working on it.” Autonomy without termination guarantees isn’t autonomy; it’s a runaway process with good manners.

So every retry path in the system carries a counter and a cap. When a finding exhausts its cap, it isn’t retried into infinity; it’s marked unfixable, and the run moves on and reports it honestly.

Table 3. Bounded retry limits and execution time budgets for autonomous workflow loops.

Listing 4

nodes.py · review_node
# This mutation lives INSIDE the node, not in the edge. (See why below.)
if current < state.max_review_reworks:
    state.review_reworks_per_finding[fid] = current + 1
else:
    state.unfixable_findings.append(fid)   # cap hit → stop trying

Three Tests per Loop. Always.

If you only test the happy path, you’ve proven the loop runs, not that it recovers, and not that it terminates. Those are three separate properties, and each one needs its own test.

• Test A (happy path): the loop never fires, nothing failed, the normal route works.
• Test B (recovery): the loop fires once, repairs the situation, then proceeds.
• Test C (bounded retry): the loop fires until the cap, then terminates cleanly into an unfixable report. Skip anyone, and you’ve left a property unproven.

Where the Agents Break

The most dangerous bug in an agentic system isn’t a crash. A crash is honest. The dangerous bug is the one where every component reports “done,” the run reports SUCCESS, and no real work happened. Four of these shipped in my system, looking exactly like success. All four are worth internalizing because they’re not specific to my code; they’re structural to how agentic pipelines fail.

Break #1 – the run that succeeds at doing nothing

The Developer fails: a timeout, a crash, nothing committed. But the graph moves on to Build anyway. The unchanged app builds fine. The run reports DONE. The target was never touched. The fix is to make failure loud: every failure-prone phase sets Phase.FAILED and appends to state.errors, and a conditional edge routes straight to a real failure report.

Listing 5

graph.py
# the bug: unconditional edge marches on regardless
g.add_edge("develop", "build")

# the fix: route on FAILED
def after_develop(state):
    return "report" if state.current_phase == Phase.FAILED else "build"

Break #2 – exit 0, zero changes, “fix applied”

Claude Code’s –print mode takes its prompt from stdin. Pass a long, multi-line prompt as a positional argument instead, and the shell mangles it. Claude Code sees empty input, does nothing, and exits 0. Every calling agent then believes the fix landed. The contract is simple once you know it, but get it wrong once and three agents (Developer, Build-Fixer, Fixer) silently no-op.

Listing 6

agents/claude_code.py
# WRONG — empty input, exit 0, no diff, fake success
subprocess.run(["claude", "--print", prompt])   # positional!

# RIGHT — safe even with braces, quotes, newlines
subprocess.run(
    ["claude", "--print", "--add-dir", repo],
    input=prompt,     # prompt over stdin
    timeout=600,
)

Break #3 – the counter that increments but never persists

This one is specific to checkpointed state machines, and it’s vicious. State changes made inside a conditional edge function don’t survive the SqliteSaver checkpoint boundary. Increment a retry counter there, and it silently resets – so the loop never reaches its cap, and you’re back to the self-funding bug. The rule that falls out: nodes write; edges decide. Every counter mutation moves into a node function; the edge only inspects the state and returns a route. A sibling of this bug: iterating all fix attempts and re-reading a stale needs_rework verdict from attempt #1 after attempt #2 already passed, routing back to FIX forever. Reason over the latest attempt per finding only. Both ran clean in tests but surfaced only in a live run.

Break #4 – the environment fights back

Two problems here, both operational. First, the host.docker.internal trap: ZAP runs in a container and reaches the app that way, while nuclei, sqlmap, and the fuzzer run on the host and need localhost. Cross the wires, and the host tool hangs 30-60 seconds before timing out, burning the time budget on nothing. Defense in depth: normalize the target URL so the prompt’s intent is guaranteed regardless of what the model emits.

Listing 7

pentester/net.py
def _normalize_target_url(url):
    # the prompt says localhost; this guarantees it
    return url.replace("host.docker.internal", "localhost")

Second, and more importantly, as a general principle: the Pentester reads HTTP responses from the app it’s attacking, and those responses are untrusted data. A malicious response body could contain instructions intended for the model driving the attack. Every tool result is wrapped in untrusted_data tags, and the system prompt forbids following anything inside them. Any agent that consumes output from a system it doesn’t control is subject to this exposure. A pentesting agent has it by definition.

Pentest as a CI stage, not a PDF

Assemble all of the above, and the payoff is a pipeline where security is a stage with a verdict, not a document that rots in a shared drive (Fig. 3).

Fig. 3: Commit → build → self-pentest → gate → ship. The gate blocks on unfixed, capped-out findings.

When the pentest becomes a gate four things change. Security runs per change, not per quarter, on the exact diff that was just written, while it’s still cheap to fix. The attacker scales with the coder; if an agent writes the code, an agent attacks it at the same volume. The output is machine-actionable verified findings, fix prompts, validated patches, and an audit trail, not a PDF that rots. And the net effect is secure velocity: AI-generated code, secured at AI speed, shipping faster precisely because the gate is automatic. A finding that hits its cap fails the gate; honestly, the loop produces a real failure report, never a fake pass.

The Disciplines that Make it Safe

If you want to build something like this (and I think within a couple of years most serious pipelines will have some version of it), the architecture matters less than the disciplines. Five of them did most of the work of maintaining an autonomous system’s trustworthiness.

1. Skeleton first. Stand up the orchestrator with stub agents and live state before any real logic. A working spine in week 2 beats a brilliant Pentester wired to nothing in week 4.
2. One job per agent. Agents never call each other. Each reads state, does one thing, writes back. Isolation is what makes a failure findable.
3. Bound every loop. A counter and a cap on every retry path. When the cap is hit, mark it as unfixable and move on – never retry indefinitely.
4. Three tests per loop. Happy path, recovery, bounded-retry. Prove it runs, prove it recovers, prove it stops. Skipping anyone leaves a property untested.
5. Make failure loud. Phase.FAILED plus conditional edges to a real failure report. The worst outcome is a green run that did nothing.

Recap

The software lifecycle is collapsing toward a single prompt that produces a whole system, and the human is moving to the edges of that loop. Security can’t stay outside it, scheduling audits against a codebase that regenerates faster than anyone can read it. It has to move inside the loop, run on every change, and, because whole-app pentesting can’t keep that pace, test what changed autonomously as a gate. Not a calendar event, a pipeline stage. There is no “left” left; put security where the code is being written.

$ git commit -m “security is now a stage”

The stack, for the curious: LangGraph + SQLite for orchestration and checkpoints; the Claude API (Opus and Sonnet) for agent reasoning; Claude Code headless for the code edits; ZAP, Nuclei, sqlmap, and a fuzzer for the actual attacks; OWASP Juice Shop in Docker as the target; and FastAPI + WebSockets for a live dashboard.

Adapted from the DevOpsCon Berlin 2026 session “There Is No ‘Left’ Left: Building a Self-Pentesting Pipeline.” Statistics: Verizon DBIR 2026, IBM Cost of a Data Breach 2025 (breach cost). Target application: OWASP Juice Shop.